Skip to content

[api][spark] Add REST permission and data policy management - #9377

Draft
JingsongLi wants to merge 8 commits into
apache:masterfrom
JingsongLi:codex/rest-permission-management
Draft

[api][spark] Add REST permission and data policy management#9377
JingsongLi wants to merge 8 commits into
apache:masterfrom
JingsongLi:codex/rest-permission-management

Conversation

@JingsongLi

@JingsongLi JingsongLi commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Purpose

Add an experimental, language-neutral REST management contract for permission assignments and table data policies, and expose it to Spark through catalog procedures. Management is provided directly by RESTCatalog without changing the generic Catalog interface.

Permission paths are /v1/{prefix}/permissions, /grant, and /revoke. Row-filter and column-mask policies are attached below /v1/{prefix}/databases/{database}/tables/{table}/policies. Here prefix is the REST Catalog server prefix, not a catalog name or catalog identifier in the permission hierarchy.

Data policies only restrict reads that the caller is already authorized to perform; they do not grant table access.

Changes

  • Add structured permission resources, principals, scopes, access types, assignments, and idempotent grant/revoke operations.
  • Add row-filter and column-mask policies backed by named functions, typed arguments, principal targeting, and exception principals.
  • Add public permission and policy methods to RESTApi, and expose permissionManagement() and policyManagement() directly from RESTCatalog.
  • Add Spark CALL procedures for granting, listing, and revoking permissions, and for creating, replacing, listing, and dropping data policies.
  • Preserve the existing PagedList contract: each list call returns one page, and an empty page terminates pagination.
  • Add stable permission ordering, inherited-permission projection, atomic assignment replacement, and unsafe-request retry protection.
  • Key policies by stable table UUID, preserve them across rename, remove them on drop, and serialize policy changes with schema and table lifecycle operations.
  • Reject schema changes that disable query authorization or invalidate policy functions, arguments, or referenced columns.
  • Add an OpenAPI 3.1 management specification, contract validation, and expanded REST/Spark usage documentation.

Tests

  • mvn -pl paimon-api -Dtest=PermissionManagementJsonTest,RESTPolicyManagementTest,RESTPermissionManagementTest,HttpClientRetrySafetyTest,RequestJacksonCompatibilityTest test
  • mvn -pl paimon-core -am -Dtest=RESTPermissionStoreTest,ResourcePathsTest -Dsurefire.failIfNoSpecifiedTests=false test
  • mvn -pl paimon-spark/paimon-spark-ut -am -DwildcardSuites=org.apache.paimon.spark.procedure.PermissionProcedureTest -Dtest=__NoSuchTest__ -Dsurefire.failIfNoSpecifiedTests=false test
  • mvn -pl paimon-spark/paimon-spark-ut -am '-Dtest=SparkCatalogWithRestTest#testRowFilter+testColumnMaskingAndRowFilter' -DwildcardSuites=__NoSuchSuite__ -Dsurefire.failIfNoSpecifiedTests=false test
  • node docs/scripts/validate-rest-openapi.js

@JingsongLi
JingsongLi marked this pull request as ready for review August 25, 2026 01:03
@JingsongLi
JingsongLi marked this pull request as draft August 25, 2026 05:25
@JingsongLi JingsongLi changed the title [api][spark] Add REST permission management [api][spark] Add REST permission and data policy management Aug 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant